<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Strict//EN">
<html>

<head>
<meta http-equiv="Content-Language" content="en-us">
<title>synthetic vision</title>
<link rel="stylesheet" type="text/css" href="../style.css">
</head>

<body>

<div align="center">
<table class=allEncompassingTable >
 <tr>
  <td >
<p><a href="../index.html" TARGET="_top"><img src="images/homeImg.png"></a></p>



<h1>Synthetic vision</h1>

<p>Synthetic vision is achieved in CoppeliaSim via <a href="visionSensors.htm">vision sensors</a>: they can perform offscreen rendering of the scene in different modes. The generated image can then be modified/processed in various ways, for instance from within a <a href="visionCallbackFunctions.htm">vision callback function</a>:</p>

<li><a href="simIM.htm?view=alphabetical">image processing with OpenCV</a> (via the <a href="https://github.com/CoppeliaRobotics/simExtIM" target="_blank">image plugin</a>)</li>
<li><a href="simVision.htm?view=category">simple image processing</a> via the <a href="https://github.com/CoppeliaRobotics/simExtVision" target="_blank">vision plugin</a></li>
<li><a href="regularApi/simTransformImage.htm">sim.transformImage</a></li>
<li><a href="regularApi/simTransformBuffer.htm">sim.transformBuffer</a></li>

<p>Following shows a vision callback function that illustrates 3 different ways to modify an acquired image:</p>
<pre class=lightRedBox>function sysCall_vision(inData)
    -- inData.handle: handle of the vision sensor
    -- inData.resolution: resolution of the vision sensor
    -- inData.clippingPlanes: near and far clipping plane of the vision sensor
    -- inData.viewAngle: the view angle if the vision sensor is in perspective operation
    -- inData.orthoSize: the size of the view if the vision sensor is in orthogonal operation
    -- inData.perspectiveOperation: whether the vision sensor is in perspective operation

    -- e.g. use very simple functions from the vision plugin (simExtVision):
    simVision.sensorImgToWorkImg(inData.handle)
    simVision.edgeDetectionOnWorkImg(inData.handle,0.2)
    simVision.swapWorkImgWithBuffer1(inData.handle)
    simVision.uniformImgToWorkImg(inData.handle,{0.5,0.25,0})
    simVision.addBuffer1ToWorkImg(inData.handle)
    simVision.workImgToSensorImg(inData.handle)
    
    -- and/or, use more powerful functions from the image plugin (simExtImage):
    if simIM then
        local imgHandle=simIM.readFromVisionSensor(inData.handle)
        local center={inData.resolution[1]/2,inData.resolution[2]/2}
        local radius=(inData.resolution[1]+inData.resolution[2])/8
        simIM.circle(imgHandle,center,radius,{255,255,0},4)
        simIM.writeToVisionSensor(imgHandle,inData.handle)
        simIM.destroy(imgHandle)
    end

    -- Or you can directly operate on the image buffer:
    local image=sim.getVisionSensorCharImage(inData.handle)
    sim.transformImage(image,inData.resolution,4)
    image=sim.transformBuffer(image,sim.buffer_uint8rgb,1,0,sim.buffer_uint8bgr)
    sim.setVisionSensorCharImage(inData.handle,image)
    
    outData={}
    outData.trigger=true -- whether the sensor should trigger
    -- filters may append packets (in packed form, use sim.packFloatTable to pack) to this table:
    outData.packedPackets={sim.packFloatTable({1,42,57})} 
    return outData
end</pre>
  
<br>
<br>

 </tr>
</table> 
</div>  
  
  
</body>

</html>
